1 link tagged with all of: chain of thought + reasoning as pathfinding + reinforcement learning
Links
The article reframes LLM reasoning as searching through token space for paths that land on a correct answer, using a verifier to define success. It argues that RL training is essentially rejection-sampling fine-tuning on the model's own successful paths, which explains why chains of thought grow longer and why individual paths can still go wrong.
- The success probability is the sum of probabilities over all paths that reach an accepted answer, so more samples (best-of-k) reliably help, even if longer single chains stop helping.
- Under a binary reward, the REINFORCE gradient reduces to rejection-sampling fine-tuning on successful paths, scaled by the success rate; longer chains emerge if successful paths tend to be longer.
- Because only final answers are verified, intermediate tokens need not be faithful, and an imperfect verifier can enable reward hacking.
reasoning as pathfinding
reinforcement learning
chain of thought